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Abstract 

We model the joint distribution of choice probabilities and decision times in binary 
choice tasks as the solution to a problem of optimal sequential sampling, where the 
agent is uncertain of the utility of each action and pays a constant cost per unit time 
for gathering information. In the resulting optimal policy, the agent’s choices are more 
likely to be correct when the agent chooses to decide quickly, provided that the agent’s 
prior beliefs are correct. For this reason it better matches the observed correlation 
between decision time and choice probability than does the classical drift-diffusion 
model, where the agent is uncertain which of two actions is best but knows the utility 
difference between them. 
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1 Introduction 


In laboratory experiments where individuals are repeatedly faced with the same choice set, 
the observed choices are stochastic—individuals don’t always choose the same item from a 
given choice set, even when the choices are made only a few minutes apartd In addition, 
individuals don’t always take the same amount of time to make a given decision—response 
times are stochastic as well. Our goal here is to model the joint distribution of choice 
probabilities and decision times in choice tasks, which we call a choice process. 

We restrict attention to the binary choice tasks that have been used in most neuroscience 
choice experiments, and suppose that the agent is choosing between two items that we call 
left (/) and right (r). In this setting we can ask how the probability of the more frequent 
(i.e., modal) choice varies with the time taken to make the decision. If the agent is learning 
during the decision process, and is stopped by the experimenter at an exogenous time, we 
would expect the data to display a speed-accuracy tradeoff, in the sense that the agent 
makes more accurate decisions (chooses the modal object more often) when given more time 
to decide. However, in many choice experiments there is instead the opposite correlation: 
slower decisions are less likely to be modal (Swensson, 1972; Luce, 1986; Ratcliff and McKoon, 
2008). 

To explain this, we develop a new variant of the drift diffusion model (DDM); other 
versions of the DDM have been extensively applied to choice processes in the neuroscience 
and psychology literatures.^ The specihcation of a DDM begins with a diffusion process Zt 
that represents information the subjects is receiving over time, and two disjoint stopping 
regions and Sf. The agent stops if at some point in time t it happens that Zt G Si 
(in which case he chooses 1) or Zt G Sf (in which case he chooses r); otherwise the agent 
continues. Because the evolution of the diffusion depends on which choice is better, the 
model predicts a joint probability distribution on choices and response times conditional on 
the true state of the world, which is unknown to the agent. 

The oldest and most commonly used version of the DDM (which we will refer to as simple 
DDM) specifies that the stopping regions are constant in time, i.e.. Si = S'' and Sf = S'', 

^See Hey (1995, 2001), Ballinger and Wilcox (1997), Cheremukhin, Popova, and Tutino (2011). 

^The DDM was first proposed as a model of choice processes in perception tasks, where the subjects are 
asked to correctly identify visual or auditory stimuli. (For recent reviews see Ratcliff and McKoon (2008) 
and Shadlen, Hanks, Churchland, Kiani, and Yang (2006).) More recently, DDM-style models have recently 
been applied to choice tasks, where subjects are choosing from a set of consumption goods presented to them. 
Clithero and Rangel (2013); Krajbich, Armel, and Rangel (2010); Krajbich and Rangel (2011); Krajbich, Lu, 
Camerer, and Rangel (2012); Milosavljevic, Malmaud, Huth, Koch, and Rangel (2010a); Reutskaja, Nagel, 
Camerer, and Rangel (2011) 
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and that Zt is a Brownian motion with drift eqnal to the difference in ntilities of the items. 
This specihcation corresponds to the optimal decision rnle for a Bayesian agent who believes 
that there are only two states of the world corresponding to whether action I or action r 
is optimal, pays a constant flow cost per nnit of time, and at each point in time decides 
whether to continne gathering the information or to stop and take an action.^ The constant 
stopping regions of the simple DDM imply that the expected amonnt of time that an agent 
will gather information depends only on the cnrrent valne of Zf, and not on how mnch time 
the agent has already spent observing the signal process, and that the probability of the 
modal choice is independent of the distribntion of stopping times. ^ In contrast, in many 
psychological tasks (Chnrchland, Kiani, and Shadlen, 2008; Ditterich, 2006) reaction times 
tend to be higher in incorrect than correct trials. For this reason, when the simple DDM 
is applied to choice data, it predicts response times that are too long for choices in which 
the stimnlns is weak, or the ntility difference between them is small. Ad-hoc extensions of 
DDM have been developed to better match the data, by allowing more general processes 
Zf or stopping regions SI, see e.g.. Laming (1968); Link and Heath (1975); Ratcliff (1978). 
However, past work has left open the qnestion of whether these generalizations correspond 
to any particular learning problem, and if so, what form those problems take. 

Our main focus in this paper is to provide learning-theoretic foundations for an alternative 
form of DDM, where the agent’s behavior is the solution to a sequential sampling problem 
with a constant cost per unit time as in the simple DDM but with a prior that allows her to 
learn not only which alternative is better, but also by how much. In this uncertain-difference 
DDM, the state of the world is the vector 6 = {d\ 6^) of the utilities of the two choice. In 
this model an agent with a large sample and Zt close to zero will decide the utility difference 
is small, and so be more eager to stop than an agent with the same Zt but a small sample. 

Our main insight is that the nature of the learning problem matters for the optimal 
stopping strategy and thus for the distribution of choices and response times. In particular, 
we show that in the uncertain-difference DDM it is optimal to have the range of Zt for which 
the agent continues to sample collapse to 0 as time goes to inhnity, and moreover that is 
does so asymptotically at rate 1/t. The intuition for the fact that the boundary should 
converge to 0 is not itself new, and has been put forward both as a heuristic in various 

^Wald (1947) stated and solved this as a hypothesis testing problem; Arrow, Blackwell, and Girshick 
(1949) solved the corresponding Bayesian version. These models were brought to the psychology literature 
by Stone (1960) and Edwards (1965). 

"*^Stone (1960) proved this independence directly for the simple DDM in discrete time. Our Theorem 1 
shows that the independence is a consequence of the stopping boundaries being constant. 
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related models and as a way to better fit the data (see, e.g., Shadlen and Kiani, 2013) but 
we provide the hrst precise statement and solution of an optimization problem that generates 
decreasing boundaries, thus providing a foundation for their use in empirical work, such as the 
exogenous exponentially-declining boundaries in Milosavljevic, Malmaud, Huth, Koch, and 
Rangel (2010b).^ We then use approximation results and numerical methods to determine 
the functional form of the boundary, thus providing guidance about what the rate of collapse 
might be expected to be. 

Finally, we investigate the consequences of allowing the flow cost to vary arbitrarily with 
time. Intuitively, if the cost decreases quickly enough, this might outweigh the diminishing 
effectiveness of learning and lead to an increasing boundary. We show that this intuition is 
correct, and more strongly that any stopping region at all can be rationalized by a suitable 
choice of a cost function. Thus optimal stopping on its own imposes essentially no restrictions 
on the observed choice process, and so it is compatible with the boundaries assumed in 
Ratcliff and McKoon (2008) and Shadlen and Kiani (2013). However, the force of the model 
derives from its joint assumptions about the evolution of beliefs and the cost function, and 
the cost functions needed to rationalize various forms of boundary may or may not seem 
plausible in the relevant applications. 

One motivation for modeling the joint distribution of decision times and choices is that 
the additional information provided by decision times can lead to models that are closer 
to the underlying neural mechanisms and may therefore be more robust.® In addition, 
as shown by Clithero and Rangel (2013), including decision times leads to better out-of- 
sample predictions of choice probabilities. In other settings than the simple choice tasks we 
study here, decision times can been used to classify decisions as “instinctive/heuristic” or 
“cognitive,” as in Rubinstein (2007) and Rand, Greene, and Nowak (2012). 

In addition to the papers cited above, our theoretical approach is closely related to the 
recent work of Woodford (2014). In his model the agent can optimize the dependence of the 
process Zt on d subject to a Shannon capacity constraint, but the stopping rule is constrained 
to have time-invariant boundaries. In our model the process Zi is exogenous but the stopping 
rule is optimal subject to a cost, so the two approaches are complementary: both models 

^Drugowitsch, Moreno-Bote, Churchland, Shadlen, and Pouget (2012) state a decision problem with two 
states (e.g. known payoff difference), a fixed terminal time, and time-dependent cost functions, and discuss 
how to use dynamic programming to numerically compute the solution. Note that even with constant costs 
the boundary is decreasing when the horizon is finite. 

®See Shadlen and Kiani (2013) and Bogacz, Brown, Moehlis, Holmes, and Cohen (2006) for discussions 
of how DDM-type models help explain the correlation between decision times and neurophysiological data 
such as neuronal firing rates. 
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feature an optimization problem, but over different spaces. 

Gabaix and Laibson (2005) and Ke, Shen, and Villas-Boas (2013) look at decisions de¬ 
rived from optimal stopping rules where the gains from sampling are exogenously specihed 
as opposed to being derived from Bayesian updating, as they are here; neither paper ex¬ 
amines the correlation between decision time and accuracy. Vul, Goodman, Griffiths, and 
Tenenbaum (2014) studies the optimal predetermined sample size for an agent whose cost 
of time arises from the opportunity to make future decisions; they hnd that for a range of 
parameters the optimal sample size is one. 

Natenzon (2013) and Lu (2013) study models with an exogenous stopping rule. They 
treat time as a parameter of the choice function, and not as an observable in its own right. 
Similarly, decision held theory (Busemeyer and Johnson, 2004) specihes an exogenous stop¬ 
ping rule; while this literature does discuss the effect of time pressure it does not treat the 
observed choice times as data. Our model makes joint predictions about decisions and re¬ 
sponse times because the stopping time is chosen optimally. These additional predictions 
provide more structure on stochastic choice and can help us develop more accurate models. 

2 Choice Processes and DDMs 

2.1 Observables 

Let A = {l,r} be the set of alternatives, which we will call left (/) and right (r). Let 
T = [0, -|-cx)) be the set of decision times—the times at which the agent is observed to state 
a choice. The analyst observes a joint probability distribution P G A(y4 x T); we call this a 
choice process. We will decompose P as 

p\t) and F{t) 

where p*(t) is probability of choosing i E A conditional on stopping at time t and F{t) = 
P{A X [0,t]) is the cdf of the marginal distribution of decision times. ^ 

It will also be useful to decompose P the other way 

F* and F\t) 

^Formally, we assume that P is a Borel probability measure on Ax T. The conditional probabilities p* (t) 
exist by Theorems 10.2.1 and 10.2.2 of Dudley (2002). 
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where P® = P({i} x T) is the overall probability of choosing i G A at any time, and 
jg gf time conditional on choosing i E A. 

2.2 Speed and Accuracy 

It is easy to define accuracy in perceptual decision tasks, since in such settings the analyst 
knows which option is ‘correct.’ However, in choice tasks the agents’ preferences are subjec¬ 
tive and may be unknown to the researcher.® One way of dehning accuracy is with respect 
to the modal choice, as we expect that the objects the agent chooses more often are in some 
sense “better;” we denote the modal choice by m, the other one by o. 

The simplest possible relationship between choices and times is no relationship at all, 
that is when the distribution of stopping times is independent of the distribution of choices. 
We will equivalently dehne this property as follows. 

Definition 1. P displays a speed-accuracy independence iff is a constant function of t 

Speed-accuracy independence is a necessary implication of the simple DDM, which we 
introduce formally in the next section. The case of independence is by nature very knife- 
edge, and is natural to relax it. In this paper, we focus on qualitatively capturing a positive 
or a negative correlation between choices and time. To do this, we introduce the following 
dehnition. 

Definition 2. P displays a speed-accuracy tradeoff iff p^{t) is an increasing function of t. 

Note that this dehnition requires that the tradeoff holds for all times t. We expect there 
to be a speed-accuracy tradeoff if the agent is learning about the options before her and is 
stopped by the experimenter at an exogenous stochastic time, as by waiting longer he obtains 
more information and can make more informed choices. But even if the agent is learning, 
the observed choice process P need not display a choice-accuracy tradeoff if the stopping 
time is not exogenous but chosen by the agent as a function of what has been learned so far. 
In this case, the agent might stop sooner when she thinks his choice is likely to be accurate, 
so the choice of a stopping time may push towards the opposite side of the speed-accuracy 
tradeoff. 

Definition 3. P displays a speed-accuracy complementarity iff is a decreasing function 
of t. 

®In some cases the analyst has a proxy of the preference in form of a separately elicited ranking, see, e.g., 
Krajbich, Armel, and Rangel (2010), Milosavljevic, Malmaud, Huth, Koch, and Rangel (2010b), Krajbich, 
Lu, Camerer, and Rangel (2012). 
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A priori we could observe both kinds of P, perhaps depending on the circumstances. 
This is indeed the case; for example, Swensson (1972) and Luce (1986) report that speed- 
accuracy complementarity is observed under normal conditions, but speed-accuracy tradeoff 
is observed when subjects are incentivized on speed; see also, Shadlen and Kiani (2013).® 

The speed-accuracy tradeoff can be equivalently expressed in terms of the monotone 
likelihood ratio property. Let P be a choice process and let /* be the density of P* with 
respect to P. Suppose that P”^ is absolutely continuous w.r.t. P°; we say that P”^ and 
F° have the monotone likelihood ratio property, denoted P™ ^mlrp F°, if the likelihood 
f^{t)/f°{t) is an increasing function. 

Though the above concepts will be useful for theoretical analysis, in empirical work time 
periods will need to be binned to get useful test statistics. For this reason we introduce two 
weaker concepts that are less sensitive to hnite samples, as their oscillation is mitigated by 
conditioning on larger sets of the form [0,t]. First, let Q^{t) := be the probability 

of choosing i conditional on stopping in the interval [0, t]. Second, we say that P™ hrst order 
stochastically dominates F°, denoted P™ ^fosd F° if F'^{t) < F°{t) for all t e T. Below, 
we summarize the relationships between these concepts. 

Fact 1. 

1. Let P be a choice process and suppose that P™' is absolutely continuous w.r.t. F°. Then 
P displays the speed-accuracy tradeoff (complementarity/independence) if and only if 

'ZmLRP ^ ^MLRP r / r — P J. 

2 . If P displays a speed-accuracy tradeoff (complementarity, independence), then Q'^if) 
is an increasing (decreasing, constant) function 

3. IfQ'^if) is an increasing (decreasing, constant) function, then F"^ (Zfosd F° (F^ i/poso 

po ^ pm _ poy 

In particular, under speed-accuracy complementarity, the expected time to choose the 
modal alternative is smaller than the expected time to choose the other alternative, i.e., 
Pf™' := tdF^{t) < tdF°(f) =: Et°. Figure 1 shows that this hts the experimental 
data of Krajbich, Armel, and Rangel (2010). 

®This could be explained if agents are stopping at random under time pressure, but are using some 
other rule under normal circumstances; for example they are following the uncertain-difference DDM, to be 
described in Section 3.3. 
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Figure 1: The dots display the data from an experiment by Krajbich, Armel, and Rangel 
(2010). (Since each alternative pair is sampled only once, we used self-reported rankings 
collected by Krajbich, Armel, and Rangel (2010) to cluster the budget sets and compute 
choice probabilities.) The blue line displays simulations of the approximately optimal barrier 
b for the uncertain-difference DDM (derived in Section 3.3) with parameters c = 0.05, Uo = 
a = 1; each simulation uses 5-10® draws and time is discretized with dt = 0.01. The expected 
time to choose I is lower than the expected time to choose r for pairs of alternatives where I 
is the modal choice; it is higher for pairs where r is modal. The red line displays predictions 
of the simple DDM for any combination of parameter values. 





The proof of Fact 1 and all other results is in the Appendix. In the next section of the 
paper we study a particular class of choice processes P, called drift-diffusion models (DDM). 
Such models are dehned by a signal process Zt and a stopping boundary bt. We characterize 
the three subclasses of DDM: speed-accuracy tradeoff, complementarity, and independence 
in terms the function bf. 

2.3 DDM representations 

DDM representations have been widely used in the psychology and neuroscience literatures 
(Ratcliff and McKoon, 2008; Shadlen, Hanks, Churchland, Kiani, and Yang, 2006; Fehr and 
Rangel, 2011). The two main ingredients of a DDM are the stimulus process Zt and a 
boundary bt. 

The stimulus process Zt is assumed to be a Brownian motion with drift 6 and variance 

Zt = 6t aBt, (1) 

where Bt is a standard Brownian motion. In early applications of DDM such as Ratcliff 
(1978) Zt was is not observed by the experimenter. In some recent applications of DDM to 
neuroscience, the analyst may observe signals that are correlated with Zt, for example the 
neural bring rates of both single neurons (Hanes and Schall, 1996) and populations of them 
(e.g., Ratcliff, Cherian, and Segraves, 2003). In the later sections we interpret the process 
Zt as a signal about the utility difference between the two alternatives. 

Suppose that the agent stops at a fixed time s and choose / if > 0 and r if < 0 (and 
bip a coin if there is a tie). Let p^{s) be the frequency of the modal choice. It is easy to see 
that if 5 > 0 then the modal choice is /, and its probability p'^{s) is an increasing function of 
the exogenous stopping time s: the process starts at Zq = 0, so if s = 0 each choice is equally 
likely. At each subsequent point in time s > 0 the distribution of Zg is M{6s, a^s); thus, the 
probability of Z^ > 0 increases. The same happens if the agent stops at a stochastic time 
T that is independent of the process Zt. Thus, an exogenously stopped process Zt leads to 
a speed-accuracy tradeoff. We will now see that if the process Z* is stopped endogenously, 
i.e., depending on its value, then this effect can be reversed. 

The canonical example of a stopping time that depends on Zt is the hitting time of a 
boundary. A boundary is a function b : M+ —)■ M. Debne the hitting time r 

r = inf{t > 0 : \Zt\ > h}, (2) 
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i.e., the first time the absolute value of the process Zt hits the boundary. Let P{S,a,b) G 
A (A X T) be the choice process induced by r and a decision rule that chooses I if Zt = bt 
and r if Zt = —bt}^ 

Definition 4. A choice process P has a DDM representation (6, a, b) if P = P(6, a, b). 

We note that the assumption that the process Zt is Brownian is an important one, as 
without it the model is vacuous. 

Fact 2. Any choice process P has a “DDM-like” representation where the stochastic process 
Zt is arbitrary. 

In particular an arbitrary P may correspond to a process Zt with jumps (or a hltration 
that is not right-continuous). However, the standard assumption in the literature is that Zt 
is Brownian. The next result characterizes the relationship between speed and accuracy in 
the class of choice processes that admit a DDM representation. 

Theorem 1. Suppose that P has a DDM representation {a,S,b). Then P displays a speed- 
accuracy tradeoff (complementarity, independence) if and only ifbt is increasing (decreasing, 
constant). 

The intuition behind the proof of this theorem is as follows: Suppose that 5 > 0 (so the 
modal action is 1), that the barrier is decreasing, and that the process stopped at time t. 
The odds that a modal decision is made in this situation are 

p\t) _ F[Zt = bt\{T = t}n{\Zt\=bt}] 
p-it) ¥[Zt = -bt\{T = t}A{\Zt\=bt}Y 

^^There are boundaries for which there is a positive probability that t = oo. This cannot happen for the 
primitive objects that we consider here, which are choice processes. Thus, we only focus on those boundaries 
that lead the agent to stop in finite tine with probability 1. This property will be satisfied in any model 
where the stopping time comes from a statistical decision problem in which never stopping incurs an infinite 
cost and the value of full information is finite. 

^^Note that the parameter a can be removed here by setting a' = 1,6' = 6/a, and b' = b/a. By a similar 
argument, 6 can be assumed to be —1,0, or 1. We nonetheless retain a and 6 here as we will use them in 
the next section to distinguish between utility and signal strength. 

^^For the Brownian motion Smith (2000) and Peskir et al. (2002) show that the distribution of hitting 
times satisfies a system of integral equations depending on the boundary. The inverse problem of finding 
a boundary such that the first hitting time has a given distribution was studied in Iscoe, Kreinin, and 
Rosen (1999) in the case Brownian Random walks and they propose a Monte Carlo algorithm for solving it 
numerically. 
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where {r = t} is the event that the process Z has not crossed the barrier before time t. 
From Bayes rule and the formula for the density of the normal distribution 

F[Z, = bt\{\Zt\ = bt}] 

F[Zt = -bt\{\Zt\ = b,}] 

which is a decreasing function of t whenever b is. Moreover, a symmetry argument using the 
Brownian bridge shows that the conditioning event {r = t} does not matter. 

Theorem 1 says that the speed-accuracy tradeoff generated by exogenous stopping can 
be reversed if stopping is endogenous, i.e., the stopping time depends on the process Zf. 

The special case of the constant boundary DDM is well known to arise as the solution 
to an optimal sampling problem, for an agent who thinks there are only two possible states 
of the world. The next section presents that model in detail, and then focuses on a related 
model in which the agent is also learning about the intensity of her preference. 

3 Optimal Stopping 

3.1 Statement of the model 

Both the simple DDM used to explain data from perception tasks and our uncertain- 
difference DDM are based on the idea of sequential learning and optimal stopping. As we will 
see, the models differ only in their prior, but this difference leads to substantially different 
predictions. In the learning model, the agent doesn’t know the true utilities, 6 = {6\ 6^) G 
but has a prior belief about them po G A(]R^). The agent observes a signal which 

as in the DDM has the form of a drift plus a Brownian motion; in the learning model we 
assume that the drift of each Z* is equal to the corresponding state, so that 

dZi = e^dt + adBl (3) 

where a is the noisiness of the signal and the processes are independent.^^ The signals 
and prior lie in a probability space (D, P, where the information that the agent 

observed up to time t is simply the paths {Z*}o<<i<t.We denote the agent’s posterior belief 
about 6 given this information by Ht- Let = E [0* | Xt] be the posterior mean for 

each i = I, r. As long as the agent delays the decision she has to pay flow cost, which for now 

^^This process was also studied by Natenzon (2013) to study stochastic choice with exogenously forced 
stopping times; he allows utilities to be correlated, which can explain context effects. 
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we assume to be constant c > 0. (Section 3.4 explores the implications of time varying cost.) 
The agent’s problem is to decide which option to take and at which time. Waiting longer 
will lead to more informed and thus better decisions, but also entails higher costs. What 
matters for this decision is the difference between the two utilities, so a sufficient statistic 
for the agent is 

Zi := Z\ - Zl = {6^ - e^)t + aV2Bt, 

where Bt = — B^) is a Brownian Motion. 

When the agent stops it is optimal to choose the option with the highest posterior ex¬ 
pected value; thus, the value of stopping at time t is maXj=; ,.Xj. The agent decides opti¬ 
mally when to stop: he chooses a stopping time r, i.e., a function r : —)■ [0, -|-oo] such that 

{t <t} E Bt for all t; let T be the set of all stopping times. Hence, the problem of the agent 
at t = 0 can be stated as 

max E max X* — cr 
rST i=l,r 

With the agent’s prior in hand, we can now define a subjective version of the relationship 
between speed and accuracy. Note that Definition 3 holds in a strong sense: For each possible 
value of 6 the agent’s choices deteriorate over time. The following definition asks only that 
this is true on average, with respect to the subjective probability of the agent. Let p"^(t,9) 
be the probability of making the modal choice when the true state is 6 and the agent stops 
at t. 

Definition 5. P displays a subjective speed-accuracy complementarity iff 

I p^{t, e)pio{9)de 
e 


(4) 


is a decreasing function of t. 

Thus, under subjective speed-accuracy complementarity, the subjective probability of the 
agent of making a correct choice is a decreasing function of time. 

3.2 Certain Difference 

In the simple DDM the agent’s prior is concentrated on two points: 9i = {9'\ 9') and 9r = 
{9', 9"), where 0" > 9'. The agent receives payoff 9" for choosing I in state 9i or r in state 9r, 

^^Following the literature, in cases where the optimum is not unique, we focus on the minimal optimal 
stopping time. 
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and & < &' for choosing r in state 6i or r in state 6*^, so she knows that the magnitnde of 
the ntility difference between the two choices is \d" — 9'\, bnt doesn’t know which action is 
better. We let /io denote the agent’s prior probability of 6*/. 

This model was hrst stndied in discrete time by Wald (1947) (with a trade-off between 
type I and type II errors taking the place of ntility maximization) and by Arrow, Blackwell, 
and Girshick (1949) in a standard dynamic programming setting. A version of the resnlt 
for the continnons-time. Brownian-signal case can be fonnd for example in Shiryaev (1969, 
2007). 

Theorem 2. With a binomial prior, there is fc > 0 such that the minimal optimal stopping 
time is T = inf{f > 0 : |/i| = k}, where It = log j. Moreover, when po = 0.5, the 

optimal stopping time has a DDM representation with a constant boundary b: 

f = inf{f > 0: \Zt\ > b}d^ 

Theorems 1 and 2 imply that the simple DDM satishes speed-accnracy independence 
and a fortiori snbjective speed accnracy independence. From the point of view of most 
economic applications, the simple DDM misses an important featnre, as the assnmption 
that the agent knows the magnitnde of the payoff difference rnles ont cases in which the 
agents is learning the intensity of his preference. At the technical level, the assnmption that 
the ntility difference is known, so there are only two states, implies that the cnrrent valne of 
the process Zt is a snfficient statistic, regardless of the elapsed time; this is why the stopping 
bonndary b in this model is constant. Intnitively, one might expect that if Zt is close to zero 
and t is large, the agent wonld infer that he is close to being indifferent between I and r and 
so stops, even thongh for the same valne of Zt the agent wonld choose to continne when t is 
small. This inference is rnled ont by the binomial prior, which says that the agent is snre 
that he is not indifferent. We now tnrn to a model with a Ganssian prior which makes snch 
inferences possible. 

3.3 Uncertain-difference DDM 

In the nncertain-difference DDM, the agent’s prior po is independent for each action and 
A7 (A'o,(Jo). Given the specihcation of the signal process (1), the posterior fit is Af{Xt,af), 

^^This is essentially Theorem 5, p. 185 of Shiryaev (2007), but our formulation is superficially more 
general, as in his model the drift only depends on the sign of the utility difference, but not on its magnitude. 
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where 


1 


(5) 


XI = 


XqCTq ^ + Zla ^ 


CXi 


-2 


+ ta 


-2 


and 


cr, = 


(Tn ^ + ta 


-2 


To gain intnition, consider the agent at time t contemplating a strategy of waiting dt 
more seconds and stopping then. The agent’s ntility of stopping now is maxi=i^r XI. If the 
agent waits, he will have a more accnrate belief and hence he will be able to make a more 
informed decision, bnt he will pay an additional cost, leading to an expected change in ntility 
of (Et maxi=i^r ~ X]:'j — cdt. The main intnition for onr resnlts is that the valne 

of the additional information gained per nnit time is decreasing in t, which leads to stopping 
regions being time-dependent. 

The following theorem states this intnition formally.^® 

Theorem 3. Let t* be the minimal optimal strategy in (4). Then 


1. There is a strictly decreasing, strictly positive k : 


-)■ 


such that 


T* = inf{f > 0: \Xl - X[\ > k{t)}. 


Moreover hmt_,.oo k(t) =0. 

2. If Xq = Xq, there is a strictly positive b : IR+ —>■ M+ such that 

T* = inf{f >0: \Z\-Zl\> ht}. 

Part (1) of the theorem describes the optimal strategy t* in terms of stopping regions 
for posterior means X\ — Xf : It is optimal for the agent to stop once the expected ntility 
difference exceeds a decreasing threshold k{t). This follows from the principle of optimality 
for continnons time processes and the shift invariance property of the valne fnnction, which 
is dne to the normality of the posterior. Intnitively, if the expected ntility difference is small 
for large t, the agent conclndes that the two items are most likely indifferent and since 
is small for large t, the expected gain from learning more is low, so the agent stops. On 
the other hand, at t = 0 even if the expected ntility difference is small, (Jq is large so the 
expected valne of learning is high and thns the agent continnes. 

^®Time-dependent stopping thresholds also arise if the cost or utility functions are time-dependent or if 
there is a fixed terminal date, see e.g. Rapoport and Burkheimer (1971) and Drugowitsch, Moreno-Bote, 
Churchland, Shadlen, and Pouget (2012). 
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Part (2) of the theorem describes the optimal strategy t* in terms of stopping regions for 
the signal process Zt := Z\ — Zl. This facilitates comparisons with the simple DDM, where 
the process of beliefs lives in a different space and is not directly comparable. 

One way to understand the difference between this model and the one from the previous 
section is to consider the agent’s posterior beliefs when 0 for large t. In the certain 

difference model, the agent interprets the signal as noise, since according to his prior the 
utilities of the the two alternatives are a fixed distance apart, so the agent disregards the 
signal and essentially starts from anew. This is why the optimal boundaries are constant in 
this model. On the other hand, in the uncertain difference model the agent’s interpretation 
oi Zt ^ ^ for large t is that the two alternatives are nearly indifferent, which prompts the 
agent to stop the costly information gathering process and make a decision right away. This 
is why the optimal boundaries are decreasing in this model. 

We do not know of a closed-form solution for the functions k and 6; however, we can 
show that, as functions of the initial variance cXq, c, and noisiness a, they have to satisfy 
the following conditions. The conditions provide useful information about the identihcation 
of the parameters of the model, and about how the predictions of the model change as 
the parameters are varied in experiments. They are also used to show that k is Lipschitz 
continuous, which simplihes the analysis of the boundary value problem, and that it declines 
with time at rate at least which is at the heart of the proof of Theorem 5 below. 

Theorem 4. The optimal solution k{t, c, cxo, a) to problem (4) is Lipschitz continuous in t 
and satisfies: 


k{t, c, (To, a) = k{0, c, at, a) for allt>0 (6) 

A;(0, c, A(To, a) = A/c(0, cA“^, (Tq, a) for all X > 0 (7) 

k{t, c, (To, Xa) = Xk{t, A“^c, A“^(To, a) for all t, X > 0 (8) 

fc(0, Ac, (To, a) > X~^k{0, c, (To, a) for all A > 0. (9) 


The hrst equality follows from the fact that an agent who starts at time 0 with prior 
M{Xq, (Tq), has the same beliefs at each time t' > t as an agent who started at time t with 
prior M^Xt, a"^). This equality is used at various steps in the proofs, including showing that 
k is Lipschitz continuous in t, which is convenient for technical reasons; it allows us to 
ignore the complications of viscosity solutions and have an exact solution to the PDE that 
characterizes the optimal stopping rule. The proofs of equalities (7) and (8) use a space-time 
change. Inequality (9) follows from a space-time change argument and the fact that more 
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information is always better for the agent. 

Using the results of Bather (1962) we can characterize the functional form for k that 
satishes the above conditions with equality, and the b that corresponds to it. 


Fact 3. Let 

( 10 ) 
( 11 ) 

Then k is the only function that satisfies (6)-(9) with equality, and b is the associated bound¬ 
ary in the signal space. Moreover, there are constants fi,T > 0 such that for all t > T 

\b{t, c, do) - b{t, c, ao)| < . _2 , ^ 

((Tq + 

Fact 3 implies that since b declines to zero at rate 1/t, this implies that b does too. Using 
numerical methods, we have found that k and b are good numerical approximations to the 
optimal boundaries k and 6.^^ 

Note that if b is decreasing, like b is, then by Theorem 1, any uncertain-difference DDM 
displays speed-accuracy complementarity. We have not been able to show this but we can 
show that the uncertain-difference DDM has the following weaker property: 


and 


k(t, c, (To, a) = 
bit, c, cTo, a) = 


2cq;2((To ^ -|- a~HY 
1 

2cq;2((Tq ^ -|- a~H) 


Theorem 5. The Gaussian DDM displays subjective speed-accuracy complementarity 

This is true because the boundary k is decreasing at the rate at least 1/\/t, which, as we 
show, follows from Theorem 4. This theorem implies that the analyst will observe a speed- 
accuracy complementarity in an experiment in which in the agent faces a series of decisions 
with states [9'^, 9^) and the values of 9 are drawn according to the agent’s prior. In partic¬ 
ular, as long as the prior is correct, speed-accuracy complementarity will hold on average; 
i.e., it will hold for the average P in a given experiment. In addition, we expect that the 
speed-accuracy complementarity should hold at least approximately if the agent’s beliefs are 
approximately correct but we have not shown this formally. Moreover, the complementarity 
can hold even across experiments as long as the distributions of the states (both objective 

^^The approximations obtained by Bather (1962) and Van Moerbeke (1974) show that b however our 
simulations and the bounds from Fact 3 indicate that b approximates the solution quite accurately, so it may 
be useful for estimation purposes. 


16 






and subjective) are close enough. That is, while we expect choice-accuracy complementarity 
to hold within a given class of decision problems, it need not hold across classes: if I and 
r are two apartments with a given utility difference 6 = 9^ — 9^^ we expect the agent to 
spend on average more time here than on a problem where I and r are two lunch items with 
the same utility difference 6. This is because we expect the prior belief of the agent to be 
domain specihc and in particular, the variance of the prior, cTg, to be higher for houses than 
for lunch items. Similarly, the complementarity can hold across subjects (indeed, the data 
in Figure 1 is a cross-section of subjects), as long as their boundaries are not too different; 
in the extreme case when one subject has a cost much lower than another subject, the hrst 
one will make choices which are longer and more accurate than the choices of the second 
subject. 

Finally, we note that the uncertain difference model is equivalent to the Chernoff (1961) 
ex post regret model, where for any stopping time r the objective function is 

Ch (t) := E - oy - 1(4>4)(9' - 9')+ - ct] ; 

that is, the agent gets zero for making the correct choice and is penalized the foregone utility 
for making the wrong choice. 

Fact 4. For any stopping time r 

Ch (r) = E [max{X|, X^} — cr] -f k, 

where k is a constant independent of r; therefore, these two objective functions induce the 
same choice process. 

Chernoff and following authors have focused on the behavior of the optimal boundary 
for very small and very large values of t. Lai and Lim (2005) say that Chernoff’s heuristic 
arguments can be adapted to provide a rigorous proof that the boundary converges to zero, 
but do not provide details. Moreover, we have not found any relevant monotonicity results 
in this literature. 

3.4 Non-Linear Cost 

In deriving the DDM representation from optimal stopping, we have so far have assumed 
that the cost of continuing per unit time is constant. We have seen that in the uncertain- 
difference model, the optimal boundary decreases due to the fact there is less to learn as time 
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goes on. One would expect that the boundary could increase if costs decrease sufficiently 
quickly. This raises the question of which DDM representations can be derived as a solution 
to an optimal stopping problem when the cost is allowed to vary arbitrarily over time. The 
next result shows that for any boundary there exists a cost function such that the boundary 
is optimal in the learning problem with normal or binomial priors. Thus optimal stopping 
on its own imposes essentially no restrictions on the observed choice process; the force of the 
model derives from its joint assumptions about the evolution of beliefs and the cost function. 

Theorem 6. Consider either the Certain or the Uncertain-Difference DDM. For any finite 
boundary b and any finite set G C M_,_ there exists a cost function d : M_|_ —)■ M such that b is 
optimal in the set of stopping times T that stop in G with probability one 

inf{f G G\ \Xt\ > ht] G argmaxE [max{X.^,— (i(r)] . 

tST 

In particular, there is a cost function such that the exponentially decreasing boundaries 
in Milosavljevic, Malmaud, Huth, Koch, and Rangel (2010b) are optimal, and a cost function 
such that there is speed-accuracy independence. 

Intuitively, the reason this result obtains is that the optimal stopping rule always takes 
the form of a cut-off: If the agent stops at time t when Xt = x, she stops at time t whenever 
\Xt\ > X. This allows us to recursively construct a cost function that rationalizes the given 
boundary by setting the cost at time t equal the expected future gains from learning. To 
avoid some technical issues having to do with the solutions to PDE’s, we consider a discrete¬ 
time hnite-horizon formulation of the problem, where the agent is only allowed to stop at 
times in a hnite set G. This lets us construct the associated cost function period by period 
instead of using smoothness conditions and stochastic calculus.^® 

4 Conclusion 

The recent literature in economics and cognitive science uses drift-diffusion models with time- 
dependent boundaries. This is helpful in matching observed properties of reaction times, 
notably their correlation with chosen actions, and in particular a phenomenon that we call 

^®The proof of the theorem relies on a result on implementable stopping times from Kruse and Strack 
(2015). In another paper Kruse and Strack (2014) generalize this result to continuous time, but as the 
absolute value is not covered by their result we can not use it here. Nevertheless, we conjecture that the 
methods used in that paper can be extended to prove the result in continuous time directly. 


18 



speed-accuracy complementarity, where earlier decisions are better than later ones. In Sec¬ 
tion 2 we showed that the monotonicity properties of the boundary characterize whether the 
observed choice process displays a speed-accuracy complementarity, or the opposite pattern 
of a speed-accuracy tradeoff. This ties an observable property of behavior (the correlation 
between reaction times and decisions) to an unobservable construct of the model (the bound¬ 
ary). This connection is helpful for understanding the qualitative properties of DDMs; it 
may also serve as a useful point of departure for future quantitative exploration of the con¬ 
nection between the rate of decline of the boundary and the strength of correlation between 
reaction times and actions. 

In Section 3 we investigated the DDM as a solution to the optimal sequential sampling 
problem, where the agent is unsure about the utility of each action and is learning about 
it as the time passes, optimally deciding when to stop. We studied the dependence of the 
solution on the nature of the learning problem and also on the cost structure. In particular, 
we proposed a model in which the agent is learning not only about which option is better, 
but also by how much it is better. We showed that the boundary in this model declines to 
zero at the rate Ijt. We also showed that any boundary could be optimal if the agent is 
facing a nonlinear cost of time. 

The analysis of our paper provides a precise foundation for DDMs with time-varying 
boundaries and establishes a set of useful connections between various parameters of the 
model and predicted behavior, thus enhancing the theoretical understanding of the model 
as well as making precise its empirical content. We expect the forces identihed in this paper 
to be present in other decisions involving uncertainty: not just in tasks used in controlled 
laboratory experiments, but also in decisions involving longer time scales, such as choosing 
an apartment rental, or deciding which papers to publish (as a journal editor). We hope 
these results will be a helpful stepping stone for further work. 
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Appendix: Proofs 


A General Results 


A.l Proof of Fact 1 


Proof of Fact 1: To prove part (1) note that by the definition of a conditional distribution 
(property (c) p. 343 of Dudley, 2002) we have = /^q^] so the density of F* is 

Pit) = Since F™' is absolutely continuous w.r.t. F°, the ratio is well defined F-almost 

('f''\ T^TTl, 

everywhere and equals This expression is increasing (decreasing, constant) if and only if 

p'^{t) is increasing (decreasing, constant). 

To prove part (2), note that by the definition of a conditional distribution we have 


^ ^ ’ F{t) F{t) 


( 12 ) 


Thus, for t < t' we have 


g'(t) > Q\t') iff 
iff 


Fit) - “ “>(t)T]F(O^F(t)] 

I[o,t]P\s)dFis) ^ J^^^,^p\s)dF{s) 
m - F{f)-F{t) ’ 


which is true if p*(-) is a decreasing function since the LHS is the average of p^ on [0, t] and the RHS 
is the average on However, the opposite implication may not hold, for example, consider 

p''{t) := {t — 2/3)^ and F(t) = f for t G [0,1]. Then p^(t) is not decreasing, but Q’'{t) is. 

To prove part (3), note that by (12) we have 


F'(t) > Fpt) iff ^ ^ iff Q'(t) >P = lim Q'(s), 

S^OO 


where we used the fact that Q^[t) + Q''{t) = 1 and F^ + F’’ = 1. Thus, if is a decreasing function, 
the RHS will hold. However, the opposite obviously doesn’t have to hold. □ 


A.2 Proof of Fact 2 


Let F be a Borel probability measure on D 
u = G n let 


Ztito) 


'0 

< +1 

-1 

\ 


A X T. Define the process Zt as follows. For any 
if t < F 

if t > F and a* = I 
if t > F and a* = r. 


Note that Zt is Borel measurable for each t, so Zt is a stochastic process on the probability space 
(D,Fq,F). Set 6 := 1 and observe that {Zt,b) is a DDM-like representation of F. 
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Note that a representation with continuous paths is possible, where 


± if a* = / 

:= S * 

l-F 


however, in this representation the filtration generated by Zi is not right-continuous at zero. □ 


A.3 Proof of Theorem 1 


Let / : be the density of the distribution of stopping times, and g : R+ x R 

the density of Zt i.e 


R 4 - be 




^ < y ~ 

y/t ~ ay/i 


y-St 


a 


Vi 


where (f>{x) = is the density of the standard normal. By Bayes rule: 


p’‘{t) = ^Zt = bt\T = t, 6, a] = 


g{t,bt)F[T = t\Zt = bt,6, a] 


p^{t) = ¥[Zt = -bt\T = t,6,a] = 


fit) 

g{t, -fet)P[r = t\Zt = -bt, S, a] 

fit) 


It follows from Zq = 0 and the symmetry of the upper and the lower barrier that 

P[r = t\Zt = bt,5,a] = P[t = t\Zt = -bt, -S,a], 


(13) 


because for any path of Z that ends at bt and crosses any boundary before t, the reflection of this 
path ends at —bt and crosses some boundary at the same time. 

The induced probability measure over paths conditional on Zt = bt is the same as the probability 
of the Brownian Bridge.^® The Brownian Bridge is the solution to the SDE dZg = — ds + adBg 
and notably does not depend on the drift 6 , which implies that 

P[r = t\Zt = -bt, -5, a] = P[r = t\Zt = -bt,5,a] (14) 

Thus, by (13) and (14) we have that 

p’'it) ^ git,bt) ^ f 4:6bt \ 
p^{t) git,-bt) 

Wlog m = I and 5 > 0; the above expression is decreasing over time if and only if bt is decreasing 
over time. □ 

^®See, e.g., Proposition 12.3.2 of Dudley (2002) or Exercise 3.16, p. 41 of Revuz and Yor (1999). 
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B The Uncertain-Difference Model 


B.l The Value Function 

Our results use on the following representation of the posterior process in the uncertain-difference 
model. 


Lemma 1. For any t > 0 


XI = K + 


a 


-1 


'0 'SO! 


^dWl 


where Wl is a Brownian motion with respect to the filtration information of the agent. 


Proof: This follows from Theorem 10.1 and equation 10.52 of Liptser and Shiryaev (2001) by 
setting a = b = 0 and A = 1, B = a. □ 

Define the continuation value as the expected value an agent can achieve by using the optimal 
continuation strategy if he beliefs the posterior means to be at time t and the variance of 

his prior equaled cJq at time 0 and the noisiness of the signal is a. 

uia.-K.{X\.,Xf} — c{t — t) . 


V{t,x^,x'^,c,ao,a) := supE(t 


Lemma 2. The continuation value has the following properties: 

1- > V{t,x\x^,c,ao,a) >max{x^x'’}. 

2. V{t, x\ x^, c, ao, a) — k = V{t, x^ — k,x^ — k, c, do, a) for every /c G M. 

3. The option value V {t, x\x'^, c, do, a) — x* is decreasing in x* for i G {I, r}. 

4- V{t,x\x^,c,ao,a) is increasing in x^ andx^. 

5. The continuation value is Lipschitz continuous in x^ and x^. 

Proof of Lemma 2 

In this proof we equivalently represent a continuation strategy by a pair of stopping times 
one for each alternative. 

Proof of 1: For the lower bound, the agent can always stop immediately and get x* or x^. For 
the upper bound, the agent can’t do better than receiving a fully informative signal right away and 
pick the better item immediately. 
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Proof of 2: Fix a continuation strategy the expected payoff equals 

E - c(min{r',r’'} - t) \ - k,Xl = - k] 


= E 




r ^ dVFi + x' - A: I + ( r ^ dfFJ + x^ - k 

Jt (Tn + sa / \ Ji o-g + so 


- c (min{r', r’'} - t) \ X‘ =x‘,X[ =. 

= E - c (min{T',T'-} - t) I X‘ = x\ XI = x'’] - k . 


Intuitively, this comes from the translation invariance of the Brownian motion, i.e., the distribu¬ 
tion of XI conditional on XI = x* — A: is the same as the distribution of XI — k conditional on 
XI = X*. As P is dehned as the supremum over all continuation strategies the result follows. 


Proof of 3: The expected difference between stopping at time t with option I and using the 
continuation strategy is 


E - c(min{r',r’'} - t) \ Xl = x\Xl = x^] - x^ 

=E - x') + (A;. - x') - c(min{T',r’'} - t) \ X\ = x\Xl = x’'] 

w-1 


=E 


a 


<t’'} 


-2 


sa 


-dITi + - x') - c (mm{T',T^} - t) \ Xl = x', A[ = . 


Note that the first part is independent of x\ and {XJ^r — x^) is weakly decreasing in xK As for 
every hxed strategy the value of waiting is decreasing the supremum over all continuation 

strategies is also weakly decreasing in xK Thus it follows that the difference between continuation 
value V{t,x\x^,c,ao,a) and value of stopping immediately on the first arm x^ is decreasing in x^ 
for every t and every 


Proof of 4: The expected value of using the continuation strategy (r , r^) equals 


E A', + a;. - c(miii{T',T"} - a) | A' = x',A[ = x" 


=E 


=E 


a 


-1 




It *^0 ^ T 
^,-1 


-awl + ^' + a;. - c (min{r', r’'} - t) \ A' = x', A^ = , 


a 




-2 


u t c'q H- sex ^ 
{,,<,.}|A'=x',A[=x 

J 


—dWl + a;. - c (min{T', r’'} - t) \ A' = x', A^ = 


which is weakly increasing in x. Consequently, the supremum over all continuation strategies 
is weakly increasing in xK By the same argument it follows that V{t,x\x^,c,ao,a) is in¬ 
creasing in x’’. 


Proof of 5: The following argument establishes that the value function is Lipschitz continuous 
with constant one. The initial belief about the mean of either option is additively separable from 
the change in belief caused by the information the agent observed after time zero. Thus changing 
the initial beliefs moves the posterior beliefs linearly. Hence, for any fixed stopping time the change 
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in initial belief can at most linearly move the posterior beliefs about the mean. Furthermore, the 
expected cost are unaffected by a change in prior beliefs. Thus, the supremum over all stopping 
times can at most be linearly affected by a change in initial belief. To see this explicitly, observe 
that 


\V{0,x'',x^,c,ao,a) -V{0,y\x^,c, ao.a 
supE 


< 


supE 


max{x* + ^ 
- supE 

r 

max{a:* 




0 (Tg ^ + so 2 
pT 

max{?/* + ' 

lo ctq d- sa ^ 
-1 


r ^^L^dw!}-ciT) 

'o CTq + sa ^ 


0 tTg ^ + so 2 


dW},x'^ 


a 


-1 


-2 


a 


-1 


0 CTg ^ + sa 


0 CTg"+ sa 2 

dlT^} 


-ma.x{y’-+ [ - —dWl,x'^+ f 

Jo ^0 *^0 


a 


-1 


0 (Jq + 5a 


rdirn 


dlF^} _ c(r) 


<\y-x^\- □ 


B.2 Proof of Theorem 3 

B.2.1 Characterization 

Note that due to the symmetry of the problem V{t,x\x^,c,ao,a) = V{t,x^,x\c,ao,a). Without 
loss of generality suppose < x^ otherwise swap x^ and x^. As Xt is a Markov process, the 
principle of optimality^® implies that the agent’s problem admits a solution of the form r = inf{t > 
0 : maxi^i^r (Xo, ct)}■ Thus, it is optimal to stop if and only if 

0 = V{t, x\x^, c, (To, a) — max{x^, x'"} = V{t, x\x^, c, (Tq, a) — x^ = V{t, x* — x’’, 0, c, (Tq, a). 

An envelope argument yields that V is continuous. Using the continuity and the monotonicity of 
V we can define the function k implicitly by 

k{t) := inf{x G M: 0 = V{t, —x, 0, c, (Tq, a)}. 

As x^ — x'’ < 0, U is monotone increasing in the second argument and V (t, x^ — x^, 0, c, (Tq, a) > 0 
by Lemma 2 we have 

{0 = U {t, x^ — x'", 0, c, (To)} = {x^ — x^ < —k{t)} = {|x^ — x^’l > k{t)} . 

Hence the optimal strategy equals r = inf{t > 0 : \X\ — Xl\ > k{t)} . 

2°Our model does not satisfy condition (2.1.1) of Peskir and Shiryaev (2006) because for some stopping 
times the expected payoff is minus infinity, but as they indicate on p. 2 the proof can be extended to our 
case. 
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B.2.2 Monotonicity 

First, we show that V{t,x\x^,c,ao,a) is decreasing in t. Note that by Doob’s optional sampling 
theorem for every fixed stopping strategy r 


E 


max{X|,X^} — CT \ Xt = {x\x^) 


= E 
= E 


max{X| - X;, 0} + X;-cT\Xt = (x^ 


max{X[ — X^, 0} — CT \ Xt = {x\ x'") 




t ■ 


Define the process Xt '■= X\ — XI, and note that 


X V'i V"!" \Ai v'F I 

/ — A. — A. — An — An + 


Jo CTr, + SO Jo CTr, + sa ^ 


^0 ctq ^ + so 


0 O'Q sex 


(15) 


where IF is a Brownian motion. Define a time change as follows: Let q{k) solve k = ^ ds. 

This implies that q{k) = ^ 2 a‘^-k ■ V’(t) = the Dambis, Dubins-Schwarz theorem 

(see, e.g., Theorem 1.6, chapter V of Revuz and Yor, 1999) Wg := (-^ij(s))<jg[o 20 -“^] ^ Brownian 

motion and thus we can rewrite the problem as 


V{t,x\x^ ,c,ao,a) = 


sup E 


= sup E 
= sup E 


max{lF-,0} - c^q(T) - q{ip{t))^ I lF/-(t) =x^ -X 
max{Wr,0} - c( [ g'(s)ds^ | lF/,(t) 

KJ^t) J 


= X — X 


2a^ 

{‘Jol - sf 


+ x^ 


max{lF-,0} “ ^ ) I 


Next, we remove the conditional expectation in the Brownian motion by adding the initial value 

2a2 


V{t,x,x'',c,ao,a)= sup E 

r>'ip{t) 


maxlWr + — x*”^ , 0} — c 


m (2^0 -«) 


rds + x^ 


Define r = t — 'ip and let wlog x* < x^, then 


V(t, x\ x^, c, (To, a) = supE 

T>0 


max{lTV — |x^ — x'’|, 0} — c 


rb(i)+r 2a^ 


'm (2^0 - 

because the current state is a sufficient statistic for Brownian motion we have 

r 2 a^ 

L (20-0 - s - V’W)^ 


ds 


+ max{x^, x^}; 


V (t, x\x^, c, do, a) = supE 
f>0 


max{lTV — |x^ — x^|,0} — c 


ds 


+ max{x^, x'’} 


Note that for every fixed strategy r the cost term is increasing in t and V’(t) and thus V (t, x\x^, c, (Tq, a) — 
max{x^,x'’} is non-increasing. 
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Finally, suppose that t < t'] then 


0 = V{t, —k{t, c, ao, a), 0, c, do, a) >V{t', —k{t, c, do, a), 0, c, do, a). 

By Lemma 2, V{t\ —k{t,c,ao,a),0,c,ao, a) > 0 and hence 0 = V{t', —k{t,c,ao,a),0,c,ao,a)- 
Hence 

k{t, c, do, a) > inf{x G M: 0 = V{t', —x, 0, c, do, a)} = k{t', c, do, a) . 


B.2.3 Positivity 

The payoff of the optimal decision rule is at least as high as the payoff from using the strategy 
that stops at time A for sure. Because the information gained over a short time period A is of 
order ez and the cost is linear, we expect that it is always worth buying some information when 
the expected utility of both options is the same. To see this formally, note that 


V(t, X, X, c, ao, a) — X = 


> 


> 


> 


supE 


max{WV, 0} — 


E 


max{fTe, 0} 




z , e dz— 


r __dJ 

i {2al-s-mY J 

2cd^ 

-n s 

(2al-s-mY \ 

r __d. 

i {2al-s-mY 



2ca^e 

(2dg - ^(tj - e)2 

2ca^e 


Ipit) ) . 


As the first term goes to zero with the speed of square root while the second term shrinks linearly 
we get that V{t,x,x,c,ao,a) — max{a:,x} > 0 for some small e > 0 and thus the agent does not 
stop when his posterior mean is the same on both options. 


B.2.4 Zero limit 

Let A:(s,c, do,a) > K* > 0 for all s > t. Consider the time t history where = XI. The 
probability that the agent never stops (and thus pays infinity costs) is bounded from below by the 
probability that the process — X^ stays in the interval [—K*, K*], 


p 

sup \Xi-X:\<K{s)\Xl = X^ 

> P 

sup \Xi-X:\<K*\Xi=Xl 


se[doo) 


sG[t,oo) 


By the above time change argument this equals the probability that a Brownian motion leaves the 
interval [—K,K] in the time from to 2dg, 


p 

sup \Xi-X:\<K*\Xl=Xl 

= P 

sup |1T,| < K*{s) 


se[t,oo) 
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This probability is non-zero. Thus, there is a positive probability the agent incurs infinite cost. 
Because the expected gain is bounded by the full information payoff, this is a contradiction. □ 


B.3 Proof of Theorem 4 

Lemma 3. 


V{0,x\x'^ ,c\,ao,a) = 


= A ^ sup E 


max < Ax -I- 


'o o-Q ^ 2 


■dMi,Xx^ 


lo CTq ^ -I- sA“2q!-2 


fdM: ^ - CT 


Proof: We have that V{0, x, x'^, cX, ao, a) equals 


max < X -I- 


supE 

T 

= A“^ supE 

T 

= A“^ supE 

r 

= supE 


i 


lo ^0 


rdiTi,x'- + 


'o CTq ^ -I- so 2 


dw: \-cXt 


max I Ax'-h/ Xx^ + f 

L Jo cjf^ + sa ^ Jo Uq 


+ sa-2 


-cA^ 


ctX^ 


max < Ax -I- 


max < Ax -I- 


Ao 


0 (Tg ^ -I- sA 2 


a 


-1 


dW';,-2,Ax" + 


dM',Ax'' 


Xa-^ ] 

9 1 


} — cX^ T 

1 J 


/O CTg 


(Tg ^ -t sA“^a“2 * 


prX^ 1 I 

1 

/ 2 - dM; 

> — cA^ r 

Jo CTg ^ -1- sA ^ 

1 J 


For the step from the second to third line apply Proposition 1.4 of Chapter V of Revuz and Yor (1999) with 
Cs '■= sA“^and Hg := , (pathwise to the integrals with limits t and tX^). In the next step we 

apply a time-change, where M* := AW ^^_2 is a Brownian motion and t' is a stopping time measurable in 
the natural filtration generated by M. □ 


B.3.1 Proof of (6) 

Note that Lemma 1 implies that for any t <t' 


rt'-t 


XI, = XI + 


a 


-1 


(uq ^ + a ‘^t) + Oi ‘^s 


dWl^g 


where Wl is a Brownian motion with respect to the filtration information of the agent. Thus, if 
the agent starts with a prior at time 0 equal to AA(Xo,iTg), then his belief at time t' is exactly 
the same as if he started with a prior at t equal to J\f{Xt,a^) where Thus, 

V{t,x\x^,c,ao,a) = V{d,x\x'^,c,at-,a) so 


fe(t, c, (To, a) = inf{x > 0: 0 = y(t, 0, —x, c, (Tq, a)} 

= inf{x > 0: 0 = P(0, 0, —x, c, at, a)} = k{0, c, at, a). □ 
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B.3.2 Proof of (7) 

By Lemma 3, V{0, x\ , cX, ao, a) equals 


= A ^ sup E 


= A sup E 


max Ax + 
max < A“^x^ + 


a 


-1 


0 ^0 ^ 

-1 


-dMi,Xx^ + 


a 


-1 


dMi,X-^x^ + 


a 


0 Ug ^ + sA 2 

r' 


'0 ^0 d- 


sa 


'0 o'o A^ + 


-dMj 

^ — CT 

dM;' 

— cX~ 


sa 


= AP(0,xiA ^,X 2 A ^,cA ^,ao/X,a). 
Thus, 


k{0,c,ao,a) = mf{x > 0: 0 = P(0,0, —x, c, cjo, a)} 

= mf{x > 0: 0 = P(0,0, —x A“^, cX~^, cro/A, a)} 

= A mf{?/ > 0: 0 = P(0, 0, —y,cX~^, uo/A, a)} = Xk{0,c A“^, ao/X, a). 

Setting ao = ao/X gives the result. 


B.3.3 Proof of (8) 

First, observe that V{t, xi, X 2 , c, ao, Aa) equals 

A“^a“^ 


□ 


= supE 

T>t 

= X sup E 

r>t 


max < xi 


I —2 I \ —2 _2 s 
't <Tq + a -^s 

-1 


dlT,\x2 + 


A ^ 


max i X ^xi + 


a 


t A^cjq ^ + a 
-1™ X-l... \-l„ \-l 


t) 


t CJq + A 2q; 2^ 

dipi, A-^X 2 + r . - -dW^ [> - (cA-^)(r - t) 


XV + 


It A ag 


a 


= XV {t, A“^xi, A“^X2, A“^c, A“^(To, a) . 

Thus, 

k{t,c,ao,Xa) = inf{x > 0: 0 = F(t, 0, —x, c, uo, Aa)} 

= inf{x > 0: 0 = F(t, 0, —A“^x, A“^c, A“^iTo, a)} 

= A inf{y > 0: 0 = F(t, 0, —?/, A“^c, A“^iTo, a)} = A A:(t, A“^c, A“^cro, a). □ 


B.3.4 Proof of (9) 

By Lemma 3, F(0, x*, x^, cA, do, a) equals 

A“^ supE 


max < Xx^ + 


/o erg + sA 


dM?,Ax'’ 



tTg ^ + sA 2 q; 2 



— CT 






















As receiving more information is always better we have that for all A > 1 


V{0,x\x^,cX,ao,a) > A ^supE 


max < Ax + 


0 ^0 ^ 


dM'.Ax’’ 


/o (Tg ^ + so 2 


dM: ^ - CT 


= A ^F(0, Ax^, Ax’',c, cro,a) 

This implies that k{t, Ac, do, a) > X~^k{t, c, do, a) for all A > 1 


k{t,Xc,ao,a) = inf{x > 0: 0 = 0, —x, Ac, do, a)} 

> inf{x > 0: 0 = ^(t, 0, —xA, c, do, a)} 

= A“^ inf{y > 0: 0 = y(0,0, —y, c, do, a)} = A“^/i:(t, c, do, a). □ 


B.3.5 Lipschitz continuity of k 

1 /f? 

Let Ae = (l + ea^^dg) < 1 and note that by definition A^ do = d^. We can thus use equations 
(6), (7) and (9) to get 

k{e, c, do, a) = fe(0, c, d^, a) = Xek{0, cX~^, do) > A^fe(0, c, do, a). 


As a consequence we can bound the difference between the value of the barrier at time zero and at 
time e from below 


/c(e, c, do, a) — A:(0, c, do, a) > (l + ea ^dg) 


1 A;(0, c, do, a). 


Dividing by e and taking the limit e —)• 0 yields that kt the partial derivative of the boundary with 
respect to time satisfies 

kt{0, c, do, a) > —2a“^dQA:(0, c, do, a). 

Since by equation (6) k(t + e, c, do, a) = k{e, c, dt, a) we have that 

fc(t+ e,c, do,a) - fc(f,c, cro,a) fc(e, c, dt, a) - fc(0, c, dt, a) 

kt{t, c, do, a) = lim- = lim- 

e->0 e e->-0 e 

= /c((0, c, dj, a) > —2Q;“^d( fc(0, c, di, a) = —2a~^(j^k{t, c, do, a) > —2a“^dgA:(0, c, do, a). 


where the last equality follows from equation (6) and the last inequality follows since k and at 
are decreasing in t. Thus, kt is bounded from below; since its upper bound is zero, k is Lipschitz 
continuous in L □ 


B.4 Proof of Theorem 5 

If the agent stops at time t at the barrier /c(t, c, do, a), his posterior belief is that the true states are 
normally distributed with 0* ~ at). Thus,the probability that the agent assigns to picking I 
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when r is optimal, conditional on stopping at t, is 


p [e^ < I Xl - XI = k{t, c, ao, a)] 


{e‘-e^)-{Xl-Xl)<-k{t,c,ao,a) 

(^01 _ 0r) _ (X' - ^ ^ _k{t, c, (To, a) 


\f2at 


\f2at 


1 


$ ( -^k{t,c,ao,a)cri 


M = k{t,c,ao,a) 

= k{t,c,ao,a) 


From the symmetry of the problem, there is the same probability of mistakenly picking r instead 
of h To show that the probability of being wrong increases over time, it remains to show that 
k{t, c, (To, ct)cr^^ is decreasing in t. We have that 


dat 


[k{0,at,c,a)at 


fccr(0, c, at, a)a^ ^ - A:(0, c, at, a)a^ ^ 


We will now show that this is equal to —3c/cc(0, at, c, ot)a‘t, which is nonnegative. To see that, we 
show that feo-(0, c, ao, a)ao = —3ckc{0, c, ao,a) + k{0, c, ao,a). Set 

/3eao = cro + e=^/3e = l + — . 

<70 

Inserting in equation. (7) gives 

k{0,c,ao Pe,Oi) = k{0,c,ao + e,a) = (3ek{0,cI3~^,ao,a) 

fc(0, c, (To + e, a) — fc(0, c, cto, a) = k{0, c/3~^, ao, a) — k(0, c, ao, a) + —k(0, c/3~^, ao, a). 

<7o 


Dividing by e and taking the limit e —)• 0 yields 


ka{0,c, (To, a) 


ka{0,c, ao,a)ao 


kc{0,c,ao,a)c 


B~^ - 1 

lim^^^- 

£->■0 e 


+ —/c(0, c, ao, a) 
<70 


d 13 1 

fcc(0, c, (To, q;)c [-3] ^ H-A:(0, c, cto, a) 

oe ao 

3 1 

—kc{0, c, (To, a)c -1- k{0, c, ao, a) 

<7o (7o 

—3 c kc{0, c, ao, a) + k{0, c, ao, a). 


B.5 Proof of Fact 4 

Let K := E[max{0^, 0^}] and fix a stopping time r. To show that 


E 


-i{xi>xr}{0^-OY-l{xr:.xi}{O^-OT-CT =E max{X', X;} - cr 


+ K, 


the cost terms can be dropped. Let D be the difference between the expected payoff from the 
optimal decision and the expected payoff from choosing the correct action, D := E [ max{X.^, X^}~\ — 
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E[max{0^, 0’’}]. By decomposing the expectation into two events, 


D = E\^l^xl<xr}iXl - max{0',0^}) + l^xr<xi}iXr - max{0',0''}) 
Plugging in the definition of X* and using the law of iterated expectations, 
D = E 


= E 

= E 

= E 


- max{0',0"}) + l{xr<xi}{W\Xr] - max{0^0'■}) 
-E[max{0',0'-}|X,]) + -E[max{0',0’'}|X,]) 

Mxl<Xr}n-{0^ - 0Y\Xr] + - On^\Tr] 


B.6 Proof of Fact 3 

We rely on Bather’s (1962) analysis of the Chernoff model, which by Fact 4 applies to our model. 
Bather studies a model with zero prior precision. Since such an agent never stops instantaneously, 
all that matters is his beliefs at t > 0, which are well defined even in this case, and given by 
XI = t~^Zl and In Section 6, p. 619 Bather (1962) shows that 

.ofiV 


M^,c.oo.E)^/t = J^ + o(i) 


which implies that 


*=(<.c.°o.^) = J^+o(X). 

Fix a > 0. By equation (8) we have k{t, c, oo, a) = aV2k{t, oo, Thus 


V2^ 


k[t, c, oo, a) = 


1 


+ 0 


fX'] 


2ca 2 ) 

This implies that there exists T, /3 > 0 such that for all f > T we have 


k[t, c, oo, a) — 


1 


2 ca 2 


< 


A 

t7/2' 


Fix (To > 0 and let s := t — a^dg This way, the agent who starts with zero prior precision 
and waits t seconds has the same posterior precision as the agent who starts with ctq and waits s 
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seconds.Thus, by ( 6 ) we have k{t,c,oo,a) = A:(s, c, do, a) 


so 


A;(s, c, (To, a) — 


2ca^{a 2s + (Tg^)2 
Finally, since b{s, c, (Tq, a) = ^(s, c, (Tq, a)(T 7 ^, we have 


< 


/3 


(s TaVg 2)7/2- 


5(s,c, (To,a) - 


2 ca^ (a + (Tg ) 


< 


_ ±_ 

{s + a%()'2)5/2 


To see that (10) and (11) hold, notice that by equations ( 6 ), ( 8 ), (7) and (9) applied in that 
order, it follows that 


k{t, c, (To, a) 


= A;(0, c, (Ti, a) = ak{0, a 
= a~‘^c~^afk{0, 1 , 1 , 1 ) = 


^c, a ^at, 1) = crtk{0, a'^ca^ ^ 

K 

cq(^((T()"2 + a~HY ’ 


, 1 , 1 ) 


where k, = /c(0,1,1,1). Since b(t,c,ao,a) = k(t,c,ao,a)a^ it follows that b(t,c,ao,a) = 
—The fact that k = 1 follows from the proof of Fact 3, as any other constant 

co^((Tq +a~^t) 2 

would result in a contradiction as t —)■ cxd. 


B.7 Proof of Theorem 6 

Let G = {tn}^^i be a finite set of times at which the agent is allowed to stop and denote by T 
all stopping times r such that t £ G almost surely. As we restrict the agent to stopping times in 
T, the stopping problem becomes a discrete time optimal stopping problem. By Doob’s optional 
sampling theorem we have that 


supE 


max.{Xl,X!^} - d{T) 


supE 

T 

supE 

T 


^max{A[-A;,A; 

l\Xi-Xl\-d{r) 


-Xl] + ]^{Xl + Xl)-d{T) 
+ 2 ^"^0 


so any optimal stopping time also solves sup,. E [| — X'!^\ — 2 (i(r)]. Define Xn = \X[^ — X^^ \ for 

all n = 1 ,..., A7. Observe that (A„)„=i^,..^ 7 v is a one-dimensional discrete time Markov process. To 
prove that for every barrier there exists a cost function which generates b by Theorem 1 in Kruse 
and Struck (2015) it suffices to prove that: 


1. there exists a constant G such that E[A„_|_i|T/„] < G{1 + A„) . 

2 . A„_|_i is increasing in A„ in the sense of first order stochastic dominance 

3. z(n, y) = E[An+i — A^ | A^ = y] is strictly decreasing in y. 
see this, observe that cr? = _2 ^ — 5 -^ = ^57 = df. 
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Condition 1 keeps the value of continuing from exploding, which would be inconsistent with a 
finite boundary. Conditions 2 and 3 combined ensure that the optimal policy is a cut-off rule. 


B.7.1 Certain-Difference DDM 

Set Zt = Zl - Zl = ((9" - e')t -b V2aBt. Then 


k 


log 


log 


f F[9 = 9i\Tt] \ _ 
[F[9 = 9r\Tt]) 

/ /r \ Zt{ 9 "- 9 ') 


_j^\ / exp(-(4«2t)-i)(Z,-(g---g0t)2 \ 

1 — /r y \exp(— — (g/' — 9”)t)‘^ J 


Denote hy pn = F[9 = 9i \ the posterior probability that I is the better choice. The 
expected absolute difference of the two choices satisfies 


= \xl - xr\ = \p49" - 9') + (1 - pnW - 


= \{2p^-l)(9"-9')\ = 2{9"-9') 


Pn 



Let ifjn ■= [Zl^ — denote the change in the signal from tn-i to tn- We have 

that the log likelihood is given by In+i = In + a~‘^{9'' — 9')^|Jn+l■ We thus have 


A„ = 2(0" - 9') 


1 -t- & ‘ 


= 2 ( 0 " - 0 ') 


z,\ln\ 


1 + elCI 


(16) 


(1) : It is easily seen that E [A„+i|jy^] < (0" — 0'), so for C big enough, E[A„_|_i|Tt„] < (7(1 -|- A„). 

(2) : To simplify notation we introduce mn = \ln\- The process {mn)n=i,...,N is Markov. More 
precisely, run+i = \mn + a~‘^{9" — 0yy^„_|_i| is folded normal with mean of the underlying normal 
distribution equal to 

run + a~‘^{9" - 9')E[tpn+i\ln] = \ln\ + a“^(0" - 0')A„(f„+i - tn) 

= mn + ~ ~ - tn) (17) 


and variance 


Var [rUn + a ^(0" - 0')V’n+i] = « ‘^{0" - 9')‘^Var [V^n+i] 

= 2a-\9" - 9')\tn+i - tn) ■ 

As argued in part (2) of the uncertain difference case, a folded normal random variable increases 
in the sense of first order stochastic dominance in the mean of the underlying normal distribution. 
As (17) increases in rUn it follows that mn+i increases in rUn in the sense of first order stochastic 
dominance. By (16) rUn = \ln\ is increasing in A„ and A„+i is increasing in rUn+i = \ln+i\ this 
completes the argument. 

(3): It remains to show that z{n, An) is decreasing in A„. As {pn)n=i,...,M is a martingale, and 
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moreover conditioning on p is equivalent to conditioning on 1 — p, we have that 
z{n, An) = E A„_|_i Pi = 2 ^^// _ g/'^ “^2 ~ 

= 2(9" - 9')E |p„+i - 2 I I= 2(9" “ S') + 2 “ 

= 2(9"-9')E[p„,.-l|ft=5i^ + l 

+ 2(6»”-6»')E 2max|^-P„+ i,o| Pi = JqJT^^ + \ “ • 

As p is a martingale we can replace Pn+i by 

z{n, An) = 2(0" - 0')E Pn-^ Pn= 2 ^g/f" + \ 

+ 2{9”-e')E 2max|^-p„+i,o| pi = ^ 

= A„ + 2(0" - 0'))IE 2max|^-p„+i,o| Pn = 2 ( 0 “^! g/) + ^ 

= 4(9" - 9')E [max 11 - p„+,. o} | p„ = 2 ( 9 , 9 ,, + 5 ■ 

The above term is strictly decreasing in An as Pn+i increases in the sense of first order stochastic 
dominance in p„ and p„ in the conditional expectation is increasing in A^. 

B.7.2 Uncertain-Difference DDM 

Let us further define Pf = 2 (Tj, — As is Normal distributed with variance j3f 

and mean Aj we have that Aj+i is folded normal distributed with mean 

I A ? 

E 2 [A 2 + 1 ] = + A2(1 - m^)) , 

where denotes the normal cdf. Thus, the expected change in delta is given by 

z{i,y) = Pi\[^e - 2yd>(— 

V vr Pi 

(1) : It is easily seen that Ej [Aj+i] < Pi^J^ + Aj. 

(2) : As Ai is folded normal distributed we have that 

r.(A.+ . <!,)=! [erf (^) + erf (^) ^ 
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Taking derivatives gives that 


d 


Pi(Ai+i < y) 




_4A^ 

e Pi — 1 


< 0. 


As Aj = \X\. — it follows that y > 0 and hence, Aj+i is increasing in A* in the sense of first 
order stochastic dominance. 

(3): The derivative of the expected change of the process A equals 

=I ) < »■ 

Hence, z is strictly decreasing in y. □ 
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